My main concerns are relative to the missing intuition and unclear explanations in some parts of the paper, as well as to provide more details in the experimental section. More specific information are given in the comments below. I would overall consider the paper as based on a novel idea. References: - Reference [10] is poorly written: rather than "jmlr" a clear statement as in reference [5] would be preferable. Section 2: - It would be nice to better clarify the notation, in particular regarding graphs, before describing the method.

The paper presents a new information-theoretic lower bound for learning neural networks. In particular, it gives an exponential statistical query lower bound that applies even to neural nets with only one hidden layer using common activation functions and log-concave input distributions. By assuming log-concavity for the input distribution, the paper proves a lower bound for a more realistic setting than previous works with applied to discrete distributions. To prove this SQ lower bound, the paper extends the notion of statistical dimension to regression problems and proves that a family of functions which can be represented by neural nets with one hidden layer has exponentially high statistical dimension with respect to any log-concave distribution. It is not clear to me if there are any novel technical ideas in the proofs, but the idea of studying the SQ complexity of neural networks in order to obtain lower bounds for a more realistic class of nets is (as far as I can tell) novel and clever.

This paper describes an extension to the PILCO algorithm (Probabilistic Inference and Learning for COntrol, a data-efficient reinforcement algorithm). The proposed algorithm applies a measurement filtering algorithm during the actual experiment and explicitly takes this measurement filtering algorithm into account during the policy learning step, which uses data from the experiment. This is an important practical extension addressing the fact that measurements are often very noisy. My intuitive explanation for this approach is that the proposed approach makes the overall feedback system more "repeatable" (noise is mostly filtered out) and therefore learning is faster (given that the filtering is effective, see last sentence of the conclusion). The paper presents detailed mathematical derivations and strong simulation results that highlight the properties of the proposed algorithm.

Sometimes we want to put a threshold function at the output of a layer. This can be for a variety of reasons. One of them is that we want to summarize the activations into binary values. This binarization of activations can be useful in autoencoders. However, thresholding poses a problem during backpropagation. The derivative of threshold functions is zero.

Originally published on Towards AI. Join over 80,000 subscribers and keep up to date with the latest developments in AI. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

This is a paper that came out in the midst of 2018, addresses the problem of scalability of searching a network architecture. These papers address the problem of Neural Architecture Search or NAS in short. As the name suggests, the idea behind this field is to explore how can we automatically search deep learning model architectures. Currently, most of the data science problems are solved by manually designing the model architecture which gives "state of the art" results on any given dataset. The problem with this approach is that, though these architectures perform really good on the standard datasets, they don't perform as expected on the organisation specific datasets.

We all know how XGBoost dominates in Kaggle competitions due to its performance and speed. This blog is about understanding how XGBoost works (try to explain the research paper). This blog is not about how to code/ implement XGboost or how to tune its hyperparameters. XGBoost stands for eXtreme Gradient Boosting. It explains bagging (bootstrap aggregating) and boosting (Adaptive Boosting).

Explaining decisions of deep neural networks is a hot research topic with applications in medical imaging, video surveillance, and self driving cars. Many methods have been proposed in literature to explain these decisions by identifying relevance of different pixels. In this paper, we propose a method that can generate contrastive explanations for such data where we not only highlight aspects that are in themselves sufficient to justify the classification by the deep model, but also new aspects which if added will change the classification. One of our key contributions is how we define "addition" for such rich data in a formal yet humanly interpretable way that leads to meaningful results. This was one of the open questions laid out in Dhurandhar et.al. (2018) [5], which proposed a general framework for creating (local) contrastive explanations for deep models. We showcase the efficacy of our approach on CelebA and Fashion-MNIST in creating intuitive explanations that are also quantitatively superior compared with other state-of-the-art interpretability methods.

What are Convolutional Neural Networks and why are they important? Convolutional Neural Networks (ConvNets or CNNs) are a category of Neural Networks that have proven very effective in areas such as image recognition and classification. ConvNets have been successful in identifying faces, objects and traffic signs apart from powering vision in robots and self driving cars. In Figure 1 above, a ConvNet is able to recognize scenes and the system is able to suggest relevant tags such as'bridge', 'railway' and'tennis' while Figure 2 shows an example of ConvNets being used for recognizing everyday objects, humans and animals. Lately, ConvNets have been effective in several Natural Language Processing tasks (such as sentence classification) as well. ConvNets, therefore, are an important tool for most machine learning practitioners today.